Skip to content

Conversation

@thuibr
Copy link
Contributor

@thuibr thuibr commented Oct 2, 2024

The self.blob.download_to_file already performs decompression per this StackOverflow post. Rather than attempting to decompress the gzip file a second time and receiving an error, allow self.blob.download_to_file to decompress it.

An alternative to this would be to use the raw_download=True kwarg for self.blob.download_to_file if that is desirable.

This closes #1456.

@thuibr thuibr changed the title prevent automatic decompression by google client Fix Double Decompression When Using GCloud GZIP Oct 2, 2024
@thuibr
Copy link
Contributor Author

thuibr commented Oct 2, 2024

@jschneier what are your thoughts on this change as opposed to using raw_download=True in the client per https://stackoverflow.com/questions/67744979/how-to-prevent-gcs-from-automatically-decompressing-objects-when-using-python-sd ?

@thuibr
Copy link
Contributor Author

thuibr commented Oct 3, 2024

@jschneier I added a test. How do I run the tests in CI?

@thuibr
Copy link
Contributor Author

thuibr commented Oct 7, 2024

@jschneier can you merge this?

@thuibr
Copy link
Contributor Author

thuibr commented Oct 22, 2024

@jschneier can you look at this?

@thuibr thuibr changed the title Fix Double Decompression When Using GCloud GZIP [gcloud] fix double decompression when using gzip Jan 3, 2025
@thuibr
Copy link
Contributor Author

thuibr commented Jan 3, 2025

@jschneier what do I need to do to get this PR looked at and approved?

@jschneier
Copy link
Owner

jschneier commented Jan 4, 2025

@thuibr can you confirm this fixes your issue? We will also need to fix CI.

@thuibr
Copy link
Contributor Author

thuibr commented Jan 6, 2025

@jschneier, I double checked just now that this fixes the issue. I also made a change to the test that I think fixes CI.

@thuibr
Copy link
Contributor Author

thuibr commented Jan 11, 2025

@jschneier do you need to run the CI?

@jschneier jschneier merged commit b79ea31 into jschneier:master Jan 11, 2025
22 checks passed
@thuibr thuibr deleted the gzip-gcs branch January 12, 2025 12:20
@markesha
Copy link
Contributor

@jschneier Do you mind making a release for this change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix GCloud GZIP

3 participants